Exclude azuresecuritylinuxagent namespace from log collection by default#1680
Exclude azuresecuritylinuxagent namespace from log collection by default#1680zanejohnson-azure wants to merge 1 commit into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| exclude_namespaces = ["kube-system","gatekeeper-system"] | ||
| # kube-system,gatekeeper-system,azuresecuritylinuxagent log collection are disabled by default in the absence of 'log_collection_settings.stderr' setting. If you want to enable kube-system,gatekeeper-system,azuresecuritylinuxagent, remove them from the following setting. | ||
| # If you want to continue to disable kube-system,gatekeeper-system,azuresecuritylinuxagent log collection keep the namespaces in the following setting and add any other namespace you want to disable log collection to the array. | ||
| # In the absense of this configmap, default value for exclude_namespaces = ["kube-system","gatekeeper-system","azuresecuritylinuxagent"] |
There was a problem hiding this comment.
thanks @zanejohnson-azure . shoudl we also add a test to ensure data from this namespace is excluded?
There was a problem hiding this comment.
tested on a cluster.
see below. when new image is deployed around 4:56 AM, we see logs from azure linux security agent is NOT collected anymore.
i also checked logs from other namespace, and confirm they are not affected by this change.
There was a problem hiding this comment.
if an cx wants to collect it, they can enable through using configmap
by changing
exclude_namespaces = ["kube-system","gatekeeper-system","azuresecuritylinuxagent"]
to
exclude_namespaces = ["kube-system","gatekeeper-system"]
| exclude_namespaces = ["kube-system","gatekeeper-system","azuresecuritylinuxagent"] | ||
| # If you want to collect logs from only selective pods inside system namespaces add them to the following setting. Provide namepace:controllerName of the system pod. NOTE: this setting is only for pods in system namespaces | ||
| # Valid values for system namespaces are: kube-system, azure-arc, gatekeeper-system, kube-public, kube-node-lease, calico-system. The system namespace used should not be present in exclude_namespaces | ||
| # collect_system_pod_logs = ["kube-system:coredns"] |
There was a problem hiding this comment.
should we add azuresecuritylinuxagent ns in this list?
| @logEnableKubernetesMetadata = ENV["AZMON_KUBERNETES_METADATA_ENABLED"]&.downcase == "true" || false | ||
| @logKubernetesMetadataIncludeFields = "podlabels,podannotations,poduid,image,imageid,imagerepo,imagetag" | ||
| @annotationBasedLogFiltering = false | ||
| @allowed_system_namespaces = ['kube-system', 'gatekeeper-system', 'calico-system', 'azure-arc', 'kube-public', 'kube-node-lease'] |
There was a problem hiding this comment.
should we add azuresecuritylinuxagent ns in this list?
| else | ||
| puts "config::configmap container-azm-ms-agentconfig for settings not mounted, using defaults" | ||
| @excludePath = "*_kube-system_*.log" | ||
| @excludePath = "*_kube-system_*.log,*_azuresecuritylinuxagent_*.log" |
There was a problem hiding this comment.
FMI, why is gatekeeper-system ns is not part of the defaults?
|
This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
3 participants
What
Adds
azuresecuritylinuxagentto the built-in default exclusion lists so logs from theazuresecuritylinuxagentnamespace (deployed by a sister team) are not collected by ama-logs out of the box.Changes
build/common/installer/scripts/tomlparser.rb@stdoutExcludeNamespacesand@stderrExcludeNamespacesdefaults now includeazuresecuritylinuxagent.@excludePathdefaults (no-configmap, parse-error, unsupported-schema-version branches) now include*_azuresecuritylinuxagent_*.log.kubernetes/container-azm-ms-agentconfig.yamlexclude_namespacesarrays and surrounding comments updated.Effect
tailinput skips opening files in this namespace viaExclude_Path(no I/O, no parsing).Caveat
The customer-supplied configmap replaces the default arrays. If a customer has an
exclude_namespacessetting in their configmap that does not includeazuresecuritylinuxagent, the namespace will still be collected. Documenting this and deciding whether to force-append is left as a follow-up.